Agent modeling is a critical component in developing effective policies within multi-agent systems, as it enables agents to form beliefs about the behaviors, intentions, and competencies of others. Many existing approaches assume access to other agents' episodic trajectories, a condition often unrealistic in real-world applications. Consequently, a practical agent modeling approach must learn a robust representation of the policies of the other agents based only on the local trajectory of the controlled agent. In this paper, we propose \texttt{TransAM}, a novel transformer-based agent modeling approach to encode local trajectories into an embedding space that effectively captures the policies of other agents. We evaluate the performance of the proposed method in cooperative, competitive, and mixed multi-agent environments. Extensive experimental results demonstrate that our approach generates strong policy representations, improves agent modeling, and leads to higher episodic returns.

Multi-agent systems often require agents to collaborate with or compete against other agents with diverse goals, behaviors, or strategies. Agent modeling is essential when designing adaptive policies for intelligent machine agents in multiagent systems, as this is the means by which the ego agent understands other agents' behavior and extracts their meaningful policy representations. These representations can be used to enhance the ego agent's adaptive policy which is trained by reinforcement learning. However, existing agent modeling approaches typically assume the availability of local observations from other agents (modeled agents) during training or a long observation trajectory for policy adaption. To remove these constrictive assumptions and improve agent modeling performance, we devised a Contrastive Learning-based Agent Modeling (CLAM) method that relies only on the local observations from the ego agent during training and execution. With these observations, CLAM is capable of generating consistent high-quality policy representations in real-time right from the beginning of each episode. We evaluated the efficacy of our approach in both cooperative and competitive multi-agent environments. Our experiments demonstrate that our approach achieves state-of-the-art on both cooperative and competitive tasks, highlighting the potential of contrastive learning-based agent modeling for enhancing reinforcement learning.

The ability of modeling the other agents, such as understanding their intentions and skills, is essential to an agent's interactions with other agents. Conventional agent modeling relies on passive observation from demonstrations. In this work, we propose an interactive agent modeling scheme enabled by encouraging an agent to learn to probe. In particular, the probing agent (i.e. a learner) learns to interact with the environment and with a target agent (i.e., a demonstrator) to maximize the change in the observed behaviors of that agent. Through probing, rich behaviors can be observed and are used for enhancing the agent modeling to learn a more accurate mind model of the target agent. Our framework consists of two learning processes: i) imitation learning for an approximated agent model and ii) pure curiosity-driven reinforcement learning for an efficient probing policy to discover new behaviors that otherwise can not be observed. We have validated our approach in four different tasks. The experimental results suggest that the agent model learned by our approach i) generalizes better in novel scenarios than the ones learned by passive observation, random probing, and other curiosity-driven approaches do, and ii) can be used for enhancing performance in multiple applications including distilling optimal planning to a policy net, collaboration, and competition. A video demo is available at https://www.dropbox.com/s/8mz6rd3349tso67/Probing_Demo.mov?dl=0

Agent modeling--the ability to model and reason other agents' knowledge, beliefs, goals, and actions--is central to intelligent interaction. The Workshop on Agent Modeling, held as part of the Thirteenth National Conference on Artificial Intelligence, was organized to bring together researchers working in these areas to assess the state of the art and discuss the common issues in representation and reasoning with models of agents. Agent modeling--the ability to model and reason about other agents' knowledge, beliefs, goals, and actions--is central to intelligent interaction, and it is being investigated in a variety of research areas, including distributed AI and multiagent systems, plan recognition, natural language discourse, intelligent tutoring, and user interfaces, as well as in related areas, such as game theory and cognitive science and psychology. The Workshop on Agent Modeling, held as part of the Thirteenth National Conference on Artificial Intelligence, was organized to bring together researchers working in these areas to assess the state of the art and discuss the common issues in representation and reasoning with models of agents. The workshop succeeded in drawing together researchers from a surprising variety of backgrounds and diverse concerns about agent modeling.

The Workshop on Agent Modeling, held as part of the Thirteenth National Conference on Artificial Intelligence, was organized to bring together researchers working in these areas to assess the state of the art and discuss the common issues in representation and reasoning with models of agents.

Interestingly, the advantage for more effective access to traditional applications of agent of modeling others is diminished global and corporate information modeling, which requires an agent to when the volatility of the domain is repositories. These repositories are model the problem-solving processes low. Thus, the models of other agents increasingly multimedia, including of the interacting human to provide are more useful in variable domains, text, audio, graphics, imagery, and video. Now attention has turned appropriate feedback. Ole Mengshoel when they are a particularly valuable guide to predict what the other rational toward the problem of processing and D. C. Wilkins's (both of University agents are going to do. and managing multiple and heterogeneous of Illinois at Urbana-Champaign) media in a principled manner, presentation focused on recognizing